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1. Introduction 

Recently there has been much interest in the study of randomly but finitely connected 
disordered spin models. In physics they have the appeal of appearing closer to 
genuinely finite dimensional systems than their fully connected mean field counterparts. 
Furthermore, finitely connected models are found to exhibit many interesting and 
complex new features which are worthy of analysis. Their equilibrium properties have 
been studied in the context of spin glasses [H 121 13 IH El IHl , error correcting codes [3 013, 
satisfiabihty problems [IHl [HI 1121 1131 , neural networks [T31[T3] and 'small world' models 
fniEl- Such analyses involve order parameter functions, which generalize the replica 
matrices of ^Hl • The finite connectivity replica symmetry breaking theory (RSB) is still 
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under development UHl 1201 US IHl 122] • It appears that virtually all studies thus far 
have concentrated on equilibrium properties or microscopic approximation schemes |23j . 
with the exception of a spherical model 

In this paper we analyze the dynamics of finitely connected disordered Ising spin 
models using the generating functional method of j2SI, which has a strong record in 
disordered spin systems, particularly in applications to systems with non-symmetric 
bonds (e.g. [211 I2Z1 I2H1 123)- Here we apply this method to randomly and finitely 
connected Ising spin models with independently drawn random bonds and synchronous 
spin updates. The random connectivity graphs are generated such as to allow for a 
controlled degree of symmetry. In the thermodynamic limit one then finds a closed 
theory, describing an effective single spin problem with dynamic order parameters which 
represent the probabilities of single-spin paths, conditional on external field paths. 

For fully asymmetric systems our equations simplify and close already for the 
single-spin path probabilities without external fields. Now one can solve various models 
completely, including phase diagrams. We work out the theory for finitely connected 
ferromagnets, and spin glass models with ±J or Gaussian random bonds. Application 
to neural networks (after adaptation of the theory, since here the bonds are no longer 
statistically independent) recovers and complements results of jSHj; which were originally 
found via counting arguments. As usual, the simplifications found for asymmetric 
dilution can be traced back to the absence of loops in the asymmetric random graph 
(see e.g. PUIEI]). 

Away from asymmetric dilution, i.e. in the nontrivial regime, we calculate the 
single-spin path probabilities for the first few time steps exactly. We also approximate 
the stationary state solution numerically by truncating the single-spin paths. This 
approximation is controlled, in that it improves systematically upon increasing the 
number of time steps taken into account. However, as the dynamics slows down near 
phase transitions and for low temperatures, the accuracy of such numerics demands 
an increasingly heavy price in CPU since the dimension of the space of probabilities 
over paths grows exponentially with the length of paths. Finally we construct explicit 
approximations for the stationary solution of our equations for the case of symmetric 
dilution, in terms of an effective field distribution, which is shown to reduce to the RS 
equilibrium theory corresponding to sequential dynamics, at least in leading nontrivial 
order in the inverse average connectivity c~^. Throughout this paper we present results 
of numerical simulations in support of our theoretical findings. 

2. Model definitions 

Our model consists of Ising spins (jj G {—1,1} on a random graph. Their dynamics 
are given by a Markovian process which describes synchronous stochastic alignment 
of the spins to local fields of the form /ij(cr;t) = J^j^iCijJijCrj + 9i(t), with 
cr = (ai, . . . , (Jat) G {—1, 1}^- Upon defining pt(cr) as the probability to find the system 
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at time t in microscopic state cr, this process can be written as 

The parameter j3 = measures the noise in the dynamics, which becomes fully 
random for /5 = and fully deterministic for (3 ^ oo. The (symmetric) bonds Jij = Jji 
are drawn independently from a probability distribution P{J). The 6i{t) define external 
(perturbation) fields. The c^- G {0, 1} specify the microscopic realization of the graph, 
and are chosen randomly and independently according to 

t<j: P{c,,) = + (1 - ^)5.,,,o (2) 

t>j: P{cij) = e4.,,c,, + (1 - e)[^4.,,i + (1 - ^)5c,,,o] (3) 

The average number of connections per spin c is assumed to remain finite in the limit 
N ^ oo. The parameter e G [0,1] controls the symmetry of the graph. The microscopic 
graph and bond variables {cij, Jij} are regarded as quenched disorder. We write averages 
over the process (H} as (■■■), averages over the disorder as and JdJ P{J)f{J) 
as {f{J))j- Detailed balance holds only for e = 1, still we will find (in theory and 
simulations) that also for e < 1 a macroscopic stationary state is approached as t oo. 



3. Generating functional analysis 



Following [25] we assume that for N oo the macroscopic behaviour of the system 
depends only on the statistical properties of the disorder (the system is self-averaging), 
and we concentrate on the calculation of the disorder averaged generating functional 



i t<t„ 



= Y.---11 PKO),...,o-(U]expHEE^,(t)a.(t)] (4) 
(T(o) cr(t„) i t 

We isolate the local fields in the usual manner via delta functions, which gives 



Z[il^] = J {dhdh} E ••• E p(o-(0))e^^[^'^>'^^>] 



cr(o) (T(tm) 

it 



JP[{o-}, {h}] = AT-i log [e-^S^^'^'WS. '^'^-^'-'"-'^] (6) 

with {dhdh} = Yln[dhi{t)dhi{t) / 271]. Upon performing the disorder average in ^ 
one finds, as always for finite connectivity models, expressions involving exponentials 
of exponentials. Site factorization in can now be achieved, provided we choose 
initializations of the form pq{(t{0)) = Y[iPo{<^i{^)), if we insert 1 = J^a ^(T,(Ti I dh 6(h — 
hi) for all i and subsequently isolate 

P{a,h■W^{t),h,{t)}) = N-'Y.^(T,(TAi^-i^^) (7) 
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Now vectors refer to paths: cr, = (crj(O), crj(l), crj(2), . . .), and similar for hj. All this 
results in an expression for Z[il)\ which can be evaluated by steepest descent: 

^{dPdP} e^*[{^'^>l (8) 



^![..]=i /"dh^P(o-,h)P(o-,h) + - /rfhrfh' ^ P(cr, h)P(o-', h')A(o-, h; o-', h' 
cr 2 cr,cr' 



+iogEpowo)) /n 



dh{t)dh{t) eMt)[h{t)-e{t)]-^pa{t+i)h{t) 
2^ 2cosh[/5/i(t)] 



with 



A{(T, h; cr', h') = (^ee-'-^{fr.i.'+(7'.K) ^ ^^^^^-i^cr.h' ^ ^-^_^y-^lcT' .i. - 2 + e)^ (10) 

(in which we have neglected contributions which will vanish for N —>■ oo, and eliminated 
the now redundant generating fields i/'). Functional variation of . .] with respect to 
P(cr, h) and P(cr, h) gives the following saddle-point equations, respectively: 

P(<T,h) =2c;E /"rfh'A(o-,h;<T',h')P(cr',h') (11) 
cr' 

P(rT',h') = ((5o-',o-5[h'-h])0 (12) 
with a measure (• • which can be interpreted in terms of an effective single spin: 

(/(^,h)),^^-^-^^^(-'^)^(-'^'^) (13) 



M{cT,h\0) =Po(^(0))e-^('^''^) /n 



dh{t) e 



ih(t)[h{t)-9{t)]+(3cr{t+l)h(t) 



27r 2cosh[/?/i(t)] 



(14) 



To infer the physical meaning of (fT^ we write the delta function over h in integral form 
and expand the exponential containing the conjugate fields. We can identify powers of 
the conjugate fields with derivatives with respect to our external field 6{t), resulting in 

Using e •^''/(x) = /(x + 6') and performing an inverse Fourier transform gives 

P{cT\e') = Jdh e-'^'-^Pia, h) = {ScT,(T')0^0' (16) 

Thus P(cr|0') is the disorder-averaged probability of finding a single-spin trajectory cr, 
given that the actual local field path d is complemented by an amount 0', similarly to 
what was found in the spherical case 

Finally we expand (fT^ in powers of P(cr, h), substitute (fTTj) . and integrate out all 
conjugate fields. This results in the following compact form of our equations: 



eP(^T,|^) + (l-e)P(cr,|0) 



k>o o<e<k {"^ <Ji 

xp(^(0))n = r^T^ (17) 

' 2 cosh(/?[^(t) + ^'(t) + Eo<.<fc ^]) 
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This equation has a clear interpretation. To calculate the probability of seeing a single- 
site path cr, first a Poissonian random number k is drawn (the number of bonds attached 
to this site). Next for all k attached sites, the associated spin paths are sampled 
according to their respective distributions, from which the path probability at the central 
site then follows (given the external fields and states of the connected spins, and taking 
into account the effective retarded self- interaction induced by connection symmetry). 



4. Fully asymmetric connectivity 
4.1. The reduced theory 

In the fully asymmetric case e = our equations ()17j) simplify considerably, and close 
already in terms of P(cr|0). The latter we will now simply denote as P{(t), so 

p(^)=p(a(o))5:^ n \[dJeP{j,)j:pi^^ 

k>0 0<e<k I (Te 




<i<k c 



2cosh(/5[^(t)+Eo<.<fc ^]) 

We next sum both sides of (jl8|) over all spin states in cr except for one, say cr(t + 1). 
This leaves an equation with only single-time spin probabilities of the form P{a{t)), 
which we subsequently write in their more conventional notation Pt{a)p. 

'''^^^^^=^^ n \Id-J^''^-J^^T.PM,)\ (19) 

fc>o ^' o<e<ky J 2cosh(/:/[6/(i:) + Lo<i!<fc ^Jj 

As with infinite connectivity, the single time distribution Pt(o") obeys a Markovian 
equation, although it is not a chain. This is a consequence of the virtual absence of loops 
in the graph for asymmetric bonds (so there is no effective retarded self-interaction). 
Using the identity Pt{(y) = |[l + c"m(t)], where m{t) = (cr(t)), we can alternatively write 
fll9|l fully as an iteration for the effective single spin magnetization: 

m(t+l)= E^2-^' n [EI^ + cremit)]] {tanhimt) + i: -^e])).H...J, (20) 

fc>0 0<e<k I o-f J 0<l<k ^ 

The calculation of the co- variances C(t,t') = {a{t)a{t')) from (fTHjl requires knowledge 
of the joint distribution P{(j(t — 1), a(t' — 1)). Here we may use, for t > t', 

P(a(t), ait')) = \[l+ m{t)a{t) + m{t')a{t') + C(t, t')a{t)a{t')] 
leading to a closed expression involving only magnetizations and co-variances: 

= E ^^-'^ n I Efl + ^Mi- 1) + ^'Mt'- 1) + cj,a[C{t- l,t'- 1)] 

fc>0 0<^<fc[cr^o-; 

X (tanh(/?[e(t - 1) + E -^^]) tanh(/?[e(t'- 1) + ^ -^[])) (21) 

Q<t<k 0<e<k ^ 

X Note that the fc = term shxiply equals e"'=e^'^^(*V2 cosh[/36'(i)]. 
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As a simple test of our results, we may work out the limit c ^ oo. We isolate in 
the right-hand sides of ()20I21|) the internal fields Vk{t) = Y.o<e<k^'^e (which exist only 
for k > 0), and calculate their lowest order moments, giving 

{Vk{t)) = -{J)jm{t) {vlit)) = {v,{t)f + -^[{J')j - {J)W{t)] 

Averaging subsequently over the Poisson distributed k gives 

i:'^{Mt)) = {j)Mt) (22) 

fc>0 '^^ 

E ^(^^W) = (^) + 0{c-') (c - oo) (23) 

fc>o 

Hence for c — ^ oo we may put v(t) {J)jm{t) in ()2()|21|1 . and find 

m{t +1) = tanh(/3[^(t) + {J)jm{t)]) C{t, t') = m{t)m{t') (24) 

(recovering the equations as derived earlier in e.g. fHOlE], with a continuous P— i>F 
transition at Tc = {J)j)- For finite c the scaling with c of coupling constants has no 
structural effects on the theory; when taking c ^ oo this is obviously no longer true. 
For instance, for (J)j = it is appropriate to re-scale Je \fcJi-, which would lead to 
the fA;(t) becoming temporally correlated zero- average Gaussian variables for c oo. 

Random bond spin models with asymmetric finite connectivity 

Let us now work out our equations and the physics they describe for asymmetric finitely 
connected spin-glasses with binary bonds, i.e. P{J') = ^{l+ri)6[J' — J] + ^{l — ri)6[J'+J] 
(with T] E [—1,1]). Here we find pUjl reducing to 

(25) 

Thus, in the absence of external fields, the stationary state magnetizations (if a 
stationary state exists) follow as the fixed-points of the non-linear map F{m): 

F{m) = E ^TT-2~^ E (1 + V^Yil - r]mf-'' tanh[^(2r - A;)] (26) 

fc>0 r=0 VJ 

Expansion of F{m) for small m gives 

F(m) = r/m y ^^2-'' V (^] \2r - k\ tanh[^|2r - k\] 
t^o k\ ^o\rJ c 

- -{7]mf E -77-2"'' E ~ + 2 - 4r) tanh[— (2r - k)] + 0{nV') (27) 

3 k>2 r=2V/ C 

Numerical evaluation of the cubic term shows it to be strictly non-positive, hence one 
only has continuous P^F transitions, occurring at the following critical value for rj: 

P^F: r^-i = y £l!^2-'=E f^l |2^-'^|tanh[^|2r-A;|] (28) 

fc>0 r=OV/ ^ 



tanh(/?[^(t) + -(2r - A;)]) 
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Figure 1. Left: phase diagram in the {ri,T/J) plane of the ±J random bond model 
on a Poissonian graph with asymmetric connectivity. Solid lines: P— s-F transition lines 
for c — 2,4,8,16,32,64 (right to left). Dashed: the continuous transition at T = rjj 
corresponding to c = oo. Right: comparison between theory and numerical simulations 
with respect to the evolution of the magnetization m, for c = 5, ry = 1, and T / J = 0.5. 
Circles: solution of f^ . Squares: simulation results for N — 16,000 spins (averaged 
over 10 runs). Time is discrete, so the line segments are only guides to the eye. 



Similarly we can inspect the possible existence of a spin-glass type state, upon putting 
m{t) 0, removing external fields, and evaluating the bond averages in (12111 . The 
resulting equation is for time-translation invariant states found to be independent of 
the time arguments, and nontrivial solutions are fixed-points of a non-linear map G{q) 
(without any dependence on rf): 

G{q) = E ^ n ( I E[l + <^^m] I tanh[^ E tanh[^ E -'^\ (29) 

fc>0 0<i<ky^ aia[ J ^ 0<i<k 0<£<k 

Numerical inspection immediately shows that this equation has no non-trivial solutions; 
in [Appendix A we give an analytical proof. We thus retain for asymmetric connectivity 



and ±J random bonds only two phases, a paramagnetic and a ferromagnetic one, 
separated by (j^Hj) . The resulting phase diagrams are shown in figure ^ for different 
values of the connectivity c. In this figure we also show a comparison between the 
evolution of m as predicted by equation ()25|) and the evolution as measured in numerical 
simulations (here with for c = 5, ?7 = 1, and T/J = 0.5). The agreement is excellent. 



In a similar fashion we can work out the consequences of our equations 



for asymmetric models with Gaussian random bonds, distributed according to P{J') = 
(27rJ^)~2e~^*^"^'~'^°^^/'^^. We may now use the fact that the sum Y,o<e<k{-Je/'^)'^e has 
become a Gaussian variable, with mean {Jo/c)J2o<e<k^e variance kJ'^/c^. In 
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Figure 2. Phase diagram in the {Jq/J,T/J) plane of the Gaussian random bond 
model on a Poissonian graph with finite asymmetric connectivity. Solid lines: P— >F 
transition lines for c = 2,4,8,16,32 (from right to left). Dashed: the continuous 
transition at T = Jq corresponding to c = oo. 



particular, in the absence of external fields the nonlinear map of which the fixed-point (s) 
give the spontaneous magnetization becomes 



= E ^rr^"'' E ( I (1 + - "^)''"' / tanh[^(^(2r - k) + zy/k)] 

(30) 

For small m one has 

F(m) = m 5] ^-^2^'= \2r - k\ j Dz iimh[—{ — \2r-k\+z^)] 

k>0 r=OV/ C J 

- livmf ^2-^ E (^) r(r - 1)(3A: + 2 - 4r) 

X [dz tanh[^(— (2r - k) + zVk)] + O(m^) (31) 
J c J 



with the abbreviation Dz = {2n) ae 2^^. Again the cubic term is found to be non- 
positive, which implies the prediction of a continuous P— >F transition at 



P _ F : 1 = y ^-^2'' y I ]\2r-k\ [Dz tanh[^(^|2r - k\ + zVk)] 

(32) 

To identify possible P^SG transitions we again inspect (j2H) for m = and without 
external fields. The time translation invariant solution represents the spin-glass order 
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Figure 3. Phase diagram of attractor neural networks with Hcbbian bonds (storing 
p = ac random patterns) and asymmetric finite connectivity, in the (a, T) plane. Both 
p and c are finite, so only the markers represent physical values; the line segments 
connecting markers are guides to the eye. The possible phases are P (paramagnetic) 
and R (pattern retrieval). The values of c shown are: c — 4 (open squares), c = 8 
(open circles), c = 16 (full squares), and c = 32 (full circles). The dashed line is the 
transition corresponding to c = oo (at T = 1 — jDz ta.nh^ {l3zy/a), see e.g. |3(J| ) . 



parameter g, and corresponds to the fixed-point of 

G(5)= n (zE[l + ^^^;?]| [Dz^Dz, tanfi[^(:^ < + ^z,)] 

k>0 0<e<k { ^ aia[ ) C J o<f<fc 

X tanli[— (^ (^e + ^iJk-Y. ^^^e + ^ E ^^^e)] (33) 
^ o<e<k y e<k y 1^ i<k 

Again tliere are no nontrivial fixed-points of G(g), and tliere is lience no P— s>SG 
transition (see [Appendix A ) . Tlie bottom line is tliat we again retain only two phases, 
a paramagnetic and a ferromagnetic one, separated now by ()32|) . The resulting phase 
diagrams are shown in figure |2l for different values of the connectivity c. 



J^-.3. Recurrent neural networks with asymmetric finite connectivity 

Our third example of an Ising spin model on a random graph with finite asymmetric 
connectivity is a recurrent Hopfield type neural network. Such systems have already 
been studied earlier [^0] (for the version with symmetric finite connectivity see |14j). 
Our objectives here are to see how earlier equations can be recovered from the present 
generating functional formalism, and to add new results (e.g. phase diagrams). 

As before our model will be of the general form (^, but now those bonds present 
will have the values Jtj = Yf^=i ^t^j, with each of the p vectors (^f , . . . , ^J^) G { — 1, 1}^ 
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denoting a (random) pattern stored. Here the different bonds are, although still 
random, no longer statistically independent, so that our equations are to be slightly 
modified. Instead of one path distribution P(cr) = limN^oo J2i^(T,a^, where 
= (<7i(0), o"j(l), (Tj(2), . . .), we will now need 2^ different path distributions, one for 
each so-called sublattice = {i\^i = where = {Q, . . They are defined as 

P^(cr) = limAT^^oo |-^^|~^ ^iei^ ^cr,(Ti- Equation (flUj) is now found to be replaced by 



X ) ...) (34) 

2 cosh(/?[^(t) + j:o<i<k 



where (/(^))^ = 2 /(^)- We define the sub-lattice magnetizations m^{t) = 

J2cr{t) '^(^)-P^('^(^))) use the general identity P^(cr(t)) = ^[1 + a(t)m^{t)] to convert 
(|34|) into the following counterpart of our previous non-linear iterative map (j2(Jj) : 



e ^c'^ 



^^it + 1) = E ^2-^- ... n Eil + w] 



k>0 \ \ 0<£<k k a-i 



X tanh(/5[^(t) + - J2 ^-^H) ) •••) (35) 

To determine the location of continuous phase transitions away from the paramagnetic 
stationary state solution = for all we expand the right-hand side of for 
small {m^ and absent external fields. Continuous bifurcations are then marked by the 
existence of non-trivial solutions for an eigenvalue problem, which upon carrying out 
suitable gauge transformations on the pattern variables, takes the shape 

"^£= E^^^(--(^£ tanh(^[^^l+ E (E^")])) •••) (36) 
k>l \ \ ^ l<i<k M If I ^ 

This eigenvalue equation is of the structural form X)^' U{$, ■ $!)m^ = Am^, solved (in a 
different context) in |321- Here we require eigenvalue 1, and we have 

u{x) = 2-^c E ^ ( • •( tanh(^[x + E (Eer)]) ) • • ) 

fc>0 \ \ ^ 0<i<k M I t I J. 

= 2"^c ^S-'^P E f ) tanh(^[x + 2r - kp\) (37) 
For each of the 2^ index subsets S C {1, 2, . . . ,p} one obtains an eigenvalue jn2] 




Uanh(^Er + 2r-M))£ 
J c 




Figure 4. Examples of the stationary overlaps m{oo), i.e. fixed-points of the iterative 
map (p?^ . Left: m(oo) as function of T, for a — 0.4 and c = 5 (circles), c = 10 
(squares) and c = 20 (diamonds). Here the transitions are predicted to occur at 
T ~ 0.52, T ~ 0.54 and T ~ 0.55, respectively. Right: m(oo) as function of a, for 
T = 0.5 and c = 5 (circles), c = 10 (squares) and c = 20 (diamonds). Here the 
transitions are predicted to occur in the intervals a £ [0.4,0.6], a S [0.4,0.5] and 
a e [0.40, 0.45], respectively (a can only take values which are multiples of c~^). 



Since lim^^o -^5 = for all index sets S, the phase transition corresponds to the highest 
temperature where the largest eigenvalue equals unity. The largest Ag is found for index 
sets of size one; since the eigenvalue A5 depends only on the size of the set S we may 
take 5 = {1} and find the following equation for the transition line: 

l = cJ2 ^2i-('=+i)P 7 + " M tanh(^[2 + 2r - {k + l)p]) (38) 

fc>0 r=0 \ ^ J ^ 

This is shown in figure El in the (a,T) plane, where a = p/c. Since both p and c are 
integers, the transition is for any given value of c marked by a discrete collection of 
points, which will only become a line for c ^ 00. We will confirm below that the 
bifurcating state is a recall state, so the two possible phases are P and R. 

We may also make a so-called condensed ansatz, implying that we restrict ourselves 
to states having a macroscopic overlap with one pattern only. Since all patterns are 
equivalent, we may put m^{t) = ^^m(t) (see e.g. [Hj). This gives self-consistent 
solutions of with for 6{t) = the recall overlap m evolving according to 

m(t + 1) = E ^ ( • •( n {1 + eMt)}t^M^ E E 

k{p 

1 

X [l + m{t)Y[l-m{t)f-'ta.n\i{^[2s + 2r -kp\) (39) 
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Figure 5. Left: evolution of recall overlaps toi (solid line) and m2 (dashed 
line), as described by equation H4()|l . following the nearly symmetrical initialization 
(TOi(0),m2(0)) = (0.50,0.49) and for control parameters {T = 0.5, c = 10, p = 4}. 
Right: comparison between theory (i.e. the nonlinear map (|39|l ) and numerical 
simulations, following the pure initial conditions mo = 0.25 (dashed: theory; dotted: 
simulations) and mg — 0.5 (solid: theory; short dashed: simulations). In both cases 
the control parameters were {T = 0.5, c = 10, p = 4}. The simulations were carried 
out with N = 128,000 and averaged over ten runs. 



This recovers the corresponding equation in jHU]. It is fairly straightforward to expand 
()H9|1 for small m{t) and show that the bifurcation corresponding to has m 7^ 0, 
which confirms that ()38|) indeed marks a P— *R transition, as claimed. Iteration of ()39p 
until stationarity allows us to find the stationary overlaps m(oo) for any given value of 
the control parameters. Examples are plotted in figure EJ both as functions of T (left) 
and as functions of a (right). The locations of the critical points, where m{oo) vanishes, 
are seen to be fully consistent with the phase diagram of figure El as they should. 

Let us finally make a mixed state ansatz where the system has a non-vanishing 
overlap with two patterns, i.e. m^{t) = C,^mi(t)+^'^m2it) (such that mi (t) = {^^m^{t)) ^ 
and m2(t) = (^^m^(t))^). Again this ansatz gives self-consistent solutions of (fH^. with 
for 6(t) = the two recall overlaps {mi, 7712} now evolving according to 

\ '"'21'' ^ -LJ / ^ A:>0 \ \ ^ Q<l<k 0<e<k t^>20<e<k 

^ ( Uo<i<k[i + el^i(t) + el"^2(t)] + Uo<e<k[i + el^i(t) - el^2(t)] \ \ \ .^Q. 

V Uo<i<k[^ + + el"^2(t)] + Uo<i<k[^ - eemiit) + e|m2(t)] J '/^ 

This can also be written in combinatorial form, by counting the various occurrences 
of specific values for {Q,^]) among the k pairs, as well as the statistics of the various 
summations over pattern components. Equation ()40|) can be used, for instance, to 
demonstrate the instability of 2-mixtures (known from fully connected models) in favour 
of pure states; see e.g. figure El (left panel). In the same figure (right panel) we 
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also compare the overlap evolution as predicted by (jSHI) to the result of carrying out 
numerical simulations, with N = 128,000 spins, and with different initial conditions. 
The agreement, although not perfect, is quite satisfactory. 



5. Arbitrary connectivity symmetry 

Equation (fT7|) is closed and exact, for arbitrary degrees of symmetry, and arbitrary 
choices of the bond distribution P{J)- For e > 0, where the connectivity is no longer 
strictly asymmetric, it is no longer possible to simplify (fTTjl in a manner similar to 
what was possible for e = 0. It closes only in the space defined by the conditional 
path probabilities P{(t\6). For continuous bonds one will have continuous fields 0, 
so even on finite time-scales the order parameter space is already infinite dimensional. 
For discrete bonds, e.g. ±J random ones, the required fields are also discrete, and 
hence the space is finite dimensional (although the dimension increases exponentially 
with time). Careful inspection of the causality structure of (fTTj) shows that if the largest 
time argument in the paths ere is t, then the distributions P{cre\j£cr/c) in the right-hand 
side of fll7|) only depend on those entries of the path vector cr with time label t — 2 at 
most. Hence every spin variable couples to the local field generated by itself at times 
up to 2 steps previously or earlier, which is indeed the time needed for the effect of a 
spin change to propagate along a bond (or multiple bonds) and return. 



5.1. Numerical solution for short times 

For short times t < tj^i^^ it is perfectly straightforward and simple to solve the 
macroscopic laws ()17j) numerically, by iteration. Especially if we restrict ourselves to 
bond distributions of the form P(J') = i(l + 7])d{J' - J) + |(1 - r])S{J' + J), our 
equations close in a finite-dimensional space. Upon defining the new order parameters 
W{(T\cr') = P{a\^), with cr' e {-1, 1}*™=''' U{0}, and writing = Jr^ we find closure 
in terms of 

k>0 0<e<k [ti=±1 ^ <Ti 

^^^^°^\n 2eosh(/3{^^(t) + l:\a\t) + Eo<.<. r,a,(t)]}) ^^^^ 

Clearly, for e = (strict asymmetry) we return to (fT^ . Examples of the result of 
iterating ()41|) numerically for = (no external fields, only the internal ones 0') 
and subsequently calculating the magnetizations m(t) = X^cr '^(^)W^('''|0)) cire shown in 
figure ini These magnetization values are tested against the corresponding measurements 
in numerical simulations, with system size = 64,000 and averaged over 20 runs. Here 
ri G {§, |} with in both cases c = 2, e = 1 and /3J = 3. We observe excellent agreement 
between theory and experiment, confirming the correctness of our basic result (fTTj) also 
away from strict asymmetry. We notice in addition the familiar macroscopic oscillations 



Dynamics of disordered spin systems on finitely connected random graphs 



14 



0.5 
0.4 
0.3 

m 

0.2 
0.1 
0.0 

0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 
t 

Figure 6. Evolution of the magnetization in the ±J spin-glass on a symmetric (e = 1) 
random finitely connected Poissonian graph, without external fields. We compare the 
predictions of the theory (points connected by line segments), to that observed in 
numerical simulations (markers), for short times and following an initial state with 
mo = 2/5. Top curve and full circles: r] = 4/5. Lower curve and open circles: rj = 2/5. 
The simulations were carried out with N ~ 64,000, and all measurements were averaged 
over 20 runs. In both scenarios c = 2 and (3 J — 3. 

which one tends to have in parallel dynamics spin systems with (partly) negative bonds. 
As expected the magnitude of these oscillations reduces with increasing values of rj 
(where the fraction of positive bonds increases); repeating these experiments for rj = 1 
(positive bonds only) would show oscillations to be absent. In view of the exponential 
growth of the number of dynamical order parameters with time, it is not feasible in 
practice to iterate beyond times of the order of magnitude shown in the figure. 

5.2. Numerical solution of the stationary state 

Detailed balance holds only for e = 1, hence only then will one be able to use 
equilibrium statistical mechanical techniques for analyzing the stationary state. Due 
to the synchronous updating of the spins ((T)), the e = 1 equilibrium state is not of a 
Boltzmann form, but involves Peretto's pseudo-Hamiltonian (which depends on the 
noise level T). Away from e = 1 the only way to obtain information on the stationary 
state is to concentrate on the stationary solution (s) of our dynamical theory (fT7|) . 

The difficulty in doing this numerically, when e > 0, lies in the need to take into 
account the entire history. Hence in practice one is forced to truncate the extent to 
which history is taken into account explicitly at some appropriate memory depth A, and 
average over those spin values assumed to be too remote to have a non-negligible effect 
in the right-hand side of (fTTj). The resulting truncated equations are iterated until the 
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1 




Figure 7. Stationary state magnetizations in the ± J spin-glass on a symmetric (e = 1) 
random finitely connected Poissonian graph, without external fields. We compare the 
predictions of the present dynamical theory with truncated paths (connected markers), 
to the values obtained from finite connectivity equilibrium replica theory based on 
Peretto's pseudo-Hamiltonian and within the RS ansatz (dashed horizontal lines), as a 
function of the memory depth A of the single spin paths. Data are shown for T — 0.5 
(upper, circles), and for T — 0.8 (lower, squares). In both cases c — 5. 

magnetization has become stationary. To speed up the equihbration process we used 
a stochastic interpretation of (fTTj) . in the spirit of population dynamics algorithms: at 
each iteration step a number k was drawn from a Poissonian distribution, upon which 
k bond strenghts Ji were selected randomly from the bond distribution P{J), and k 
A-step spin trajectories cr^ were drawn (given that truncation was carried out at A steps 
into the past) from the distribution P{(T(\J^(t / c). The new distribution P{(t\0') was 
then calculated according to 

Pncw(cr|0') = ^oid(o-|0') + A. TT = r^T^ (42) 

*=J-\+i 2cosh(/3[^(t) + < + ^^'(t)]) 

where A is a small positive number. We subsequently normalized the new distribution 

-Pnew(c|^')5 3.nd moved to the next iteration step. In figured we present some results of 

the above numerical procedure for a fully symmetrically diluted ferromagnet, i.e. e = 1, 

with c = 5 and for two different temperature values T = 0.5 and T = 0.8. We truncated 

the spin paths after up to A = 10 past iteration steps. The reason for choosing e = 1, i.e. 

the detailed balance limit, is that it allows for a convenient comparison with predictions 

obtained within equilibrium theory (the finite connectivity ensemble analysis based on 

the Peretto pseudo-Hamiltonian, following JH])- In the latter theory one can obtain 

explicit independent predictions for the equilibrium magnetization, at least within the 

RS ansatz and upon solving for the various effective field distributions using standard 
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population dynamics algorithms. The result is figure [7| which gives an indication of the 
extent to which memory is to be taken into account (fT7j) . which is seen to increase as 
one approaches the critical temperature. It also confirms the correctness of our theory 
in the stationary state, complementing the short-time validation of figure IHl 



5. 3. Structural properties and approximate stationary solution at e = 1 

Finally we show how one might go beyond numerical analysis of our equations, and 
obtain both a better understanding of the structural properties of (fTTj) as well as explicit 
approximate stationary state solutions. For simplicity we send initial and final times to 
minus and plus infinity, respectively, we choose zero external fields, and we investigate 
the following ansatz for a stationary state, in terms of an effective field distribution: 

To compactify our notation we will abbreviate Htl^ J2a{t)=±i]f{<^) = {f{(^))(T- We insert 
the ansatz into the right-hand side of (fTTj) . which gives 

-c k I r . _ I3a,(t+l)[ht + i^] 

RHS= n /dMJ.P(J.)W^(Mn ^jMt),, 



JO<l<k 



^ 2 cosh(/3[e(t) + Eo<.<. ^]) la.,„a, ^^^^ 

Since all complications of the e > dynamics stem from the dependence of P{cri\^cr) 
on cr, we next try to concentrate all in exponentials using the simple identity 



\^ A( h\ ma,b)- A(a,6) = ^cosh(/3[a+&])cosh(/?[a-6]) 



2/3 

This allows us to write in the form 



B{a,b) = ^iog --2sma±m 



cosh(/3[a-b]) 



(45) 



RHs=E^ n /dMw.)w^(M}n — n A(h'^^) — 



? \2cosh{mt)+J:o<i<k^])U„^^ ^^^^ 

We note that |[cr(t — 1) + crit + 1)] G { — 1, 0, 1}. In order to transport also the {cr(t)} 
occurrences in the last line to exponentials, we use the following identity: 

C = /(O) 

(47) 



S e {-1,0, 1} : fiS) = Ce^DS,-pPS^^ D = l^\og 

F = 

Application to our present problem gives 



/(I) 1 




/(-1)J 






-1) 



RHS=5:^ n { [dhdJePiJe)W{h)]Y[ 

k>o ^- o<e<k ^ ^ 



Ui<kAihe,^) 
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t 

with the form factors 



I3a{t+l) 



^(*)-E^<fc^(''«''T)+l^'=.t[{^.^}] + 5^fe.t+2[{ft,J}] + |i^fc,t[{^J}]'7(t-l) 



DkAih. J]] 



,2cosh(/?[^(t) + Eo<^<fc ^]), 



2/3 



log 



ax...cjk 



2/3 



log 



(Tl...O-fc 



CTl...O-fc 



(4^ 



(49) 



(50) 



(Ti...(Tfe 



;5i) 



Expression (j4g|) is still fully exact, but involves potentially time-dependent form factors 
and a retarded self-interaction. We now use PH|) for constructing an approximate 
stationary solution of equation (fT7j) for large c. In [Appendix B| we derive 



^}] = - E ^^tanh[/3/i,] + 0(i) 



F,,,[{/i,J}]=0(-) (52) 



(obviously, alternative choices for the scaling with c of the bonds would lead to different 
expressions). Causality would have been violated in (pH|l as soon as Dt\{h, J}] were to 
depend on t\ it is thus satisfactory to see in that d{t) indeed drops out (the next 
order of Dt\{h^ J}] is again found to be independent of t\ In combination, if we also 
use explicit normalization, this results in the following approximated solution of ()17|): 

„ „/3(7(t+l)[0(t) + /l] 



W{K) 



E 



e ^c'^ 



fe>0 

X b 



2cosh(/3[^(t) + /i]) 



n \ fdhidJeP{Je)W{he 



0<l<k 



2 1 

h ^ J£ tanh[Phe\ + 77;^ E 



i<k 



2P 



Kk 



cos\i{(3[h, + ^]) 
cosh(/3[/i,-^]) 



+ o(- 



(54) 



The last equation ()54|) can be rewritten as 



1^(/.) = ^ 



e ^c^ 



k>0 

X 5 



n 

o<e<k 

k 



dhedJeP{Je)Wihe 



1 /^J 1 

/i - - ^ arctanh[tanh(— ) tanh(/?/i^)] + C(-) 



(55) 



In leading order in this is identical to the replica symmetric equilibrium solution of 
the sequential dynamics version of our model, as derived in |3] (on the basis of [T3] one 
expects the RS equilibrium solutions of sequential and parallel dynamics to be identical). 
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Solutions of the simple form or similar, if they exist, are expected to be typical of 
parallel as opposed to sequential dynamics. We realize that the above analysis as yet 
falls short of leading to exact solutions of our macroscopic equations, but it does suggest 
possibilities for deriving approximate solutions in a controlled manner. The latter could 
then also possibly be employed for e < 1, where equilibrium analysis is not an option. 

6. Discussion 

In this paper we used the generating functional analysis methods of De Dominicis to 
analyze the dynamics of finitely connected Ising spin models with parallel dynamics, 
random bonds, and controlled degrees of connectivity symmetry. We have derived an 
exact equation, valid in the infinite system size limit, for the dynamic order parameter 
of our problem. This order parameter represents the probability for finding a single-site 
path in configuration space, given a (finite) deviation in the local external field at that 
site. It generalizes the dynamic order parameters usually found for disordered systems 
with full or with diverging random connectivity, viz. correlation- and response functions. 

We have applied our dynamical theory first to models with strictly asymmetric 
connectivity. Here there is no effective retarded self-interaction in the problem, and 
our theory consequently simplifies (for instance, one never finds spin-glass states). 
Applications of the resulting dynamical equations include finitely connected random 
bond models (exhibiting continuous ferromagnetic phase transitions), and finitely 
connected recurrent neural network models (exhibiting recall transitions). Numerical 
simulations support our findings and predictions. Next we turned to models with 
(partly) symmetric connectivity, where the order parameter equations are much more 
complicated. We first showed how our equations can be solved iteratively for the first few 
time-steps (although the computation required grows exponentially with time, which 
limits what can be done in practice), and how the resulting predictions find perfect 
confirmation in numerical simulations. The stationary state solution of our dynamical 
theory was approximated both numerically (by truncating the effective memory of the 
non-Markovian macroscopic equations) and analytically (upon making a simple ansatz 
in the language of effective field distributions). In the latter case we had to resort to an 
approximation, which is correct in leading non-trivial order in c~^, and which up to that 
order reproduces the self-consistent equation which was found earlier for the equilibrium 
effective field distribution in RS approximation. 

We now have an exact dynamical theory for finitely connected random bond Ising 
spin models, albeit in the form of equations which are generally hard to solve (which, 
given past experience with statics and dynamics of disordered systems, will not come 
as a surprise). In this paper we also hope to have shown that solution, under certain 
conditions and/or in special limits, is nevertheless not ruled out either. Moreover, 
the availability of an exact macroscopic theory is vital for the systematic development 
of practical approximations, and also to serve as a yardstick against which to test 
alternative (and perhaps simpler) dynamical theories with the ambition of exactness. 
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Appendix A. Absence of spin-glass phase for asymmetric connectivity 

Here we show analyticaUy that for asymmetric connectivity, i.e. e = 0, there cannot be 
a spin-glass phase. The spin-glass order parameter q e [0, 1] is to be solved from the 
fixed-point equation G{q) = q, where 



G{q) = E 

k>0 



u n tE(1 + ?^^^^) IdJiPiJi 
o<e<k ' ^ 



X tanh[— ^ Je<^i] tanh[— ^ JeO'i] 



(A.l) 

e<k e<k 

We note that G{0) = 0, and that G{q) < 1 for all q G [0, 1]. We prove the absence of 
non-trivial fixed-points of G{q) by showing that G"{q) > for q > 0, which immediately 
implies that G{q) < g for < g < 1. Working out the second derivative of G{q) gives 



e '^c^ 



k>2 y'^ - o<e<k 



n 

2<e<k 



1 /9 ^ ' 

- E ^1^2 tanh[- JeCTe] 



1 (3 



(A.2) 



Here we need the objects ij{S) and i^iS'), where S = ^ J2e=3 Je'^e and S' = ^ I]f=3 Ji<^'f 



i'i^) = 7 E cri(72tanh[S' + -(Jidi + J20"2)] 
4 — r 



(A.3) 



1 (tanh[S] + tanh[f (Ji(7i+J2fX2)]) (l - tanh[5] tanh[f ( Jkji+Js^xs)] 
4 1 - tanh2[5] tanh^if ( JiCTi-fJsCTs)] 

' tanhfSl f l-tanh^[f(Ji + J2)] 

2 ^ Hl-tanh2[S]tanh2[^(Ji + J2) 



l-tanh^[f(Ji- J2)] 
1 - tanh2[5] tanh2[^(Ji - J2)] 



tanh[^](l-tanh2[S]) 


tanh^ff 


(Ji- J2)]-tanh2[f(Ji + Js)]' 




2 


1 - tanh^[^] tanh2[f (Ji + J2)]' 




"1 - tanh2[S] tanh2[|(Ji - J^)] 



It follows that 

^(5)^(5') = tanh[^] tanh[^']iy(|5|, |^'|) (A.4) 

in which the function lydSI, |S"|) is srictly non-negative and invariant under 
permutation of its arguments. Since S and S' are zero-average but positively correlated 
random variables for q > 0, one concludes that G"{q) > 0. 

Appendix B. Evaluation of form factors Dk,t[{h, J}] and Fk,t[{h, J}] 

Here we calculate the form factors (j50|l and (j5T|l for large c, where we know that in the 
Poissonian sums the physics will be dominated by those terms with k = 0{c). Both 
(fSn|) and involve averages over {ai, . . . , 0"^}, with p{ai, . . . , ak) = 2~^, and can be 
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written in the following form: 

Dk,t[{h,J}] = ^^ogm/(l>{-l)] (B.l) 

FkAih, J}] - ^ {log[0(l)/0(O)] + log[0(-l)/0(O)]} (B.2) 



where 



Hence 



2u/3 



dydy e^yy^^-y /f,Y.e<u'^^\^^-^^' 
2ti 2cos\i\3d{t)+y\ \ i ^....^^ 

J ^ 2cosm9(t)+y] n - 



e<k 



- [[cosh[fJhe\. J 2^ 2coshmt) +yf 

= n cosh[«.2,o3h[/?(^(t) + iE,<. J.tanh[«)] ^^"^^ 

^}] = - E -^^tanhf/^/i,] + (B.4) 
^ e<k ^ 

F,A{h,J}] ^0(-J (B.5) 

Working out higher orders in is in principle straightforward. Including 0{c~^) would 
convert the result of the y integration in our representation of the function from a 
5-distribution for y into a Gaussian integral, which can in turn be done analytically. 



